Maximum covariance method for weight initialization of multilayer perceptron network
نویسندگان
چکیده
منابع مشابه
IDIAP Technical report
Proper initialization is one of the most important prerequisites for fast convergence of feed-forward neural networks like high order and multilayer perceptrons. This publication aims at determining the optimal value of the initial weight v ariance (or range), which is the principal parameter of random weight initialization methods for both types of neural networks. An overview of random weight...
متن کاملAccurate Initialization of Neural Network Weights by Backpropagation of the Desired Response
Proper initialization of neural networks is critical for a successful training of its weights. Many methods have been proposed to achieve this, including heuristic least squares approaches. In this paper, inspired by these previous attempts to train (or initialize) neural networks, we formulate a mathematically sound algorithm based on backpropagating the desired output through the layers of a ...
متن کاملAccurate Initialization of Neural Network Weights
Abstracf Proper initialization of neural networks is critical for a successful training of its weights. Many methods have been proposed to achieve this, including heuristic least squares approaches. In this paper, inspired by these previous attempts to train (or initialize) neural networks, we formulate a mathematically sound algorithm based on backpropagating the desired output through the lay...
متن کاملBifurcations in Nonlinear Networks Initialized with Linear Network Weight Solutions
It has been reported [1] that weight convergence of multilayer perceptron networks (MLP) can be improved by smoothly changing the node nonlinearity from a linear to a sigmoidal function during training. While this approach might provide a useful training heuristic, formally, this method depends on an underlying homotopy which transforms a linear to a nonlinear network of the same architecture. ...
متن کاملInteractive initialization of the multilayer perceptron
A new multilayer preceptor initialization method is proposed and compared experimentally with a traditional random initialization method. An operator maps training-set vectors into a two-variate space, inspects bi-variate training-set vectors and controls the complexity of the decision boundary. Simulations with sixteen real-world pattern classi®cation tasks have shown that in small-scale patte...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1996